Complex electronic, electromechanical and mechanical products and equipment are generally tested using automated test systems. Such tests can include validation tests which run through the various operations that the device under test (DUT) is capable of and records whether each operation was performed properly; environmental tests which expose the DUT to various combinations of temperature, pressure and humidity, and records the results; production tests, etc. Generally, both the DUT and the systems providing the environmental and other constraints on the DUT are controlled electronically. In the last decade or so, computerized programs which are capable of controlling a variety of automated tests, referred to in the art as “test executive” programs, have been developed.
Test executive programs in the prior art include internal test executive programs developed by Agilent Technologies and TESTSTAND software developed by National Instruments Corporation, which is described as a ready-to-run test executive program for organizing, controlling, and executing automated prototype, validation, or production test systems. The prior art Agilent Technologies programs did not use a graphical user interface (GUI), therefore limiting the ability of the program to display large amounts of data in a simple fashion. The TESTSTAND software, while using a GUI, requires the user to scroll through multiple windows to determine the overall progress of a test.
Tests usually are defined by a set of rules or specifications to which the DUT is compared. The rules or specifications generally comprise various inputs defined by electrical and mechanical parameters applied to the DUT, such as voltage, current, specified manipulations of controls and device parts, as well as environmental parameters under which the test is conducted, such as temperature, humidity, pressure, and the time period over which a parameter is applied. Each test will include many combinations of the parameters applied to each element of the DUT, and often will be repeated many times. Each combination of parameters will define a measurement that results in one or more datapoints, which are recorded and compared to numerical or Boolean limits defining the specifications. Thus, as equipment and products become more complex, electronic test programs have become very long and complex, often requiring several days, or even a week or more to run a complete test.
Likewise, setting up connections to the DUT is also a long and complex process. There are many things that can fail or go wrong during the set-up process, both with the set-up of the connections to the DUT and with the test system itself. Failures occurring in the connections to the DUT and in the test system itself may also occur during the testing process. Failures occurring either during the set-up process or during the testing process will affect the datapoints that are recorded during the testing process. In particular, failures will affect the value of the datapoints produced by the test. If the effect on a datapoint is significant enough, comparison of that datapoint to the numerical or Boolean limits defining the specifications may show that datapoint to be outside of the limits defined by the specifications. Without knowing that a failure has caused the datapoint to fall outside of the limits defined by a specification, the datapoint will be erroneously treated as having failed to meet the specification. Likewise, if a failure causes the value of a datapoint to move within the limits defined by a specification, the datapoint will be falsely treated as having passed the specification. Such failures of the system, as opposed to failures of the DUT, are referred to herein as “erroneous results”.
Prior art test systems are not able to alert the user to erroneous results. Such prior art systems are also unable to alert the user to a significant number of marginal results, which is indicative of an inferior device. Instead, in the case of such a failure, the test progresses and the values of the datapoints are recorded, whether or not they have been affected by the failure. When those datapoints are compared to the numerical or Boolean limits defining the specifications, the results of the comparison are likewise affected by the failure. In later reviewing the results of the test, it may be possible for a user to determine that an erroneous result has occurred through abnormalities in the results of the test. However, in actual practice, the user almost always will permit the test to run unattended for hours, overnight, or for days, while other work is attended to. Because of the length and complexity of the tests, and the inferiority of the display technology of prior art test systems, when the user returns to check on the test, it takes considerable time and effort to review the results to determine the progress and results of the test and to further ascertain whether a system failure has occurred and has affected the results of the test. In many cases, it often happens that the time necessary to do a thorough analysis of the results is not available as the test is being run and the results are only reviewed after the test has been completed. As a result, much test time is wasted when test results are reviewed after a test is completed, and it is found that certain elements of the test were improperly set up, that a failure of the test system occurred during the test, or the interface with the DUT was faulty in some respect that was not recognized during the test.
The above problems lead to inefficiencies that add to the cost of products and slow product development. Thus, a test executive system that overcomes these problems would be highly desirable.